The present invention is related to video data storage. More particularly, the present invention is related to video display systems and frame buffers. Several related technologies are discussed below (in labeled sections for clarity).
1. Raster-Scan Displays
A common type of graphics monitor is a conventional raster-scan display using a cathode ray tube (xe2x80x9cCRTxe2x80x9d). As is well known, in a typical CRT, an electron beam strikes phosphor on the inner surface of the screen producing light visible on the outer surface of the screen. By controlling the electron beam different locations of the screen can be struck, creating a pattern and hence a video image. In a typical CRT raster-scan display, the screen area is divided into a grid of pixels (or picture elements). The electron beam sweeps from left to right across the screen, one row at a time from top to bottom, progressively drawing each pixel on the screen. Each row of pixels is commonly referred to as a scan line. In this type of conventional display, the scan lines are horizontal. The number of pixels in a single scan line is referred to as the width. One complete pass over the screen and the pixels in that pass are commonly referred to as a frame. As the electron beam moves across the pixels of each scan line, the beam intensity can be adjusted to vary the light produced by the screen phosphor corresponding to the pixels. The light emitted by the phosphor of the pixels creates a pattern of illuminated spots forming the video image. The intensity of the electron beam is controlled by image data stored in a section of memory called the frame buffer or refresh buffer.
2. Grating Light Valves
Another type of display system uses one or more grating light valves (xe2x80x9cGLVxe2x80x9d) to produce an image. GLV""s are known devices, and a description can be found in (among other sources) a paper by D. M. Bloom of Silicon Light Machines, Inc., titled xe2x80x9cThe Grating Light Valve: revolutionizing display technologyxe2x80x9d (1997; available from Silicon Light Machines; and a copy of which has been filed in an Information Disclosure Statement for this application), and in an article (and therein cited references) by R. W. Corrigan and others of Silicon Light Machines, Inc., titled xe2x80x9cAn Alternative Architecture for High Performance Displayxe2x80x9d (presented at the 141st SMPTE Technical Conference and Exhibition, Nov. 20, 1999, in New York, N.Y.), the disclosures of which are incorporated herein by reference. In overview, a GLV uses a combination of reflection and diffraction of light to create an image. A GLV includes a one-dimensional array of GLV pixels, each GLV pixel including a number of microscopic xe2x80x9cribbons.xe2x80x9d The ribbons for each GLV pixel can be deflected through electrostatic force to create an adjustable diffraction grating. In a non-deflected state, the ribbons reflect light. As the ribbons are deflected, the ribbons increasingly diffract light. Accordingly, by controlling the ribbons, the proportion of light that is either reflected or diffracted can be controlled for each GLV pixel. The GLV deflects the ribbons for each GLV pixel according to image data, such as pixel data received from a frame buffer.
An array of GLV pixels can create a column of visible pixels, such as 1088 pixels, typically an entire column at a time. A GLV can be used to create a vertical column of pixels in a high definition resolution image, such as a screen resolution of 1920 pixels horizontally by 1080 pixels vertically (with some of the 1088 pixels left blank or dark). By providing a GLV with pixel data representing columns of pixels in a frame, the GLV can create the frame of pixels, one column at a time, sweeping from left to right. The location of each column of pixels can be controlled external to the GLV array, such as through lenses and an adjustable mirror, rather than moving the GLV itself. A combination of three GLV""s for red, green, and blue can be used to produce a color image.
3. Frame Buffers
FIG. 1A is a representation of a screen 105 as a grid of pixels 110. In FIG. 1A, for simplicity, screen 105 is only 4xc3x974 and so only 16 pixels are shown, but a typical screen has many more pixels. One common screen resolution is high definition (xe2x80x9cHDxe2x80x9d) resolution, where screen resolution indicates the number of pixels in a frame and is typically given as the horizontal resolution (number of pixels in one row) versus the vertical resolution (number of pixels in one column). HD resolution is either 1920xc3x971080 (2,073,600 total pixels per frame) or 1280xc3x97720 (921,600 pixels per frame). Herein, HD resolution refers to 1920xc3x971080.
Returning to FIG. 1A, the pixels 110 are often numbered sequentially for reference. Pixel 0 is typically at the upper left. FIG. 1B is a representation of a memory device 150 implementing a frame buffer as a grid of memory locations 155. Typical memory devices include SDRAM (synchronous dynamic random access memory). The actual memory device used may vary in different devices, but the memory locations for the frame buffer are typically in a contiguous block of locations with sequential addresses. Memory device 150 has a memory location 155 for storing pixel data (e.g., an intensity value) for each pixel 110 of screen 105. In some implementations, pixel data for more than one pixel is stored at each memory location. In many conventional raster-scan systems, pixel data is stored in memory locations adjacent to one another in the same pattern as the pixels on the screen. In FIG. 1B, each memory location 155 is numbered with the number of the pixel (110 from FIG. 1A) corresponding to the pixel data stored in that memory location 155. For example, the pixel at the upper left of the screen is pixel 0 in FIG. 1A and pixel data for pixel 0 is stored in the first memory location in memory device 150, as indicated by the xe2x80x9c0xe2x80x9d in the upper left memory location 155. The second memory location stores pixel data for pixel 1, the fifth memory location stores pixel data for pixel 4, and so on.
4. Pixel Rates
FIG. 2 is a representation of screen resolutions and typical data throughput requirements. FIG. 2 shows four resolutions in respective areas: VGA resolution (640xc3x97480) 205, XGA resolution (1024xc3x97768) 210, SXGA resolution (1280xc3x971024) 215, and HD resolution (1920xc3x971080) 220. The pixel rate for a screen resolution is the number of pixels per second that need to be processed to maintain the screen resolution at a specified refresh rate (i.e., the number of times a complete frame is drawn to the screen per second). While pixel rates vary among implementations, the pixel rates shown in FIG. 2 are representative. These pixel rates are given in megapixels per second (xe2x80x9cMP/Sxe2x80x9d). For example, according to SMPTE 274M-1998 (a specification defining, among other things, pixel rates for resolutions of 1920xc3x971080), for HD resolution 220 the pixel rate is about 150 MP/S @60 Hz. FIG. 2 also shows a corresponding approximate data rate in megabytes per second (xe2x80x9cMB/Sxe2x80x9d) for each resolution.
The data rate is the number of bytes per second to be processed based on the number of bytes per pixel and the pixel rate. For example, HD resolution 220 has a data rate of 450 MB/S, at 24 bits per pixel (3 bytes). If each pixel has 32 bits of data, the data rate for HD resolution is 600 MB/S. However, the data rate of a typical 32-bit wide SDRAM running at 125 MHZ is approximately 500 MB/S. A frame buffer architecture using two 125 MHZ SDRAM""s can realize a data rate of approximately 1000 MB/S.
5. Frame Buffers Using Parallel Storage in Two Memory Devices
FIG. 3A is a representation of a frame 305 of pixels 310 divided between two memory devices. Frame 305 has only 32 pixels for simplicity, but, as noted above, a typical HD resolution frame has 2,073,600 pixels. FIG. 3B is a representation of a first memory device 350 and FIG. 3C is a representation of a second memory device 375. Each pixel 310 in frame 305 is numbered, starting with pixel 0 in the upper left of frame 305. Even-numbered pixels are stored in first memory device 350 and odd-numbered pixels are stored in second memory device 375. The pixels stored in second memory device 375 are also shaded for clarity in FIGS. 3A and 3C.
FIG. 4 is a block diagram of a typical frame buffer architecture 400 capable of accessing pixel data for two pixels in parallel, supporting the representations shown in FIGS. 3A, 3B, and 3C. A video source 405 provides pixel data to a first memory 410 (recall first memory device 350 in FIG. 3B) and to a second memory 415 (recall second memory device 375 in FIG. 3C) in parallel and a video destination 420 retrieves pixel data from first memory 410 and from second memory 415 in parallel. In this implementation, pixel data for each pixel is stored in a separate addressable memory location. Video source 405 receives video data from another source (not shown), such as a broadcast source or a software application running on a computer system connected to video source 405. Video destination 420 controls the display of each pixel on a video device (not shown), such as a CRT. First memory 410 and second memory 415 are separate memory devices such as two SDRAM""s. A first data bus 425 is connected to video source 405, first memory 410, and video destination 420. A second data bus 430 is connected to video source 405, second memory 415, and video destination 420. A source address bus 435 is connected to video source 405 and a first input 440 of an address multiplexor 445. A destination address bus 450 is connected to video destination 420 and a second input 455 of address multiplexor 445. An output 460 of address multiplexor 445 is connected to first memory 410 and second memory 415. Accordingly, the same address is provided to both first memory 410 and second memory 415. Address multiplexor 445 receives a control signal (not shown) to cause first input 440 or second input 455 to connect to output 460. First memory 410 and second memory 415 also receive control signals (not shown) to control whether memories 410 and 415 will read in data (write mode) or read out data (read mode). In addition, while clock lines are not shown in FIG. 4, architecture 400 operates based on clock cycles so that pixel data can be processed for two pixels per clock cycle in support of the desired pixel rate.
In operation, memories 410 and 415 read in or store complementary halves of a frame of pixels as pixel data from video source 405 and output the pixel data to video destination 420. To store pixel data, memories 410 and 415 are put in write mode and address multiplexor 445 is set to connect first input 440 to output 460. Video source 405 provides pixel data for a first pixel to first data bus 425, such as pixel 0 in FIG. 3A, and pixel data for a second pixel to second data bus 430, such as pixel 1 in FIG. 3A. First data bus 425 provides its pixel data to first memory 410 and second data bus 430 provides its pixel data to second memory 415. Video source 405 also provides an address to source address bus 435. To calculate the address, video source 405 can use a counter. Because each memory 410 and 415 stores pixel data for half the pixels in one frame, the counter typically ranges from 0 to one less than one-half of the number of pixels in one frame. Video source 405 can increment the counter by 1 for each pixel pair. Source address bus 435 provides the address to first input 440 of address multiplexor 445. Address multiplexor 445 in turn provides the address to first memory 410 and second memory 415. First memory 410 stores the pixel data on first data bus 425 at the address supplied by address multiplexor 445 from video source 405. Second memory 415 stores the pixel data on second data bus 430 at the same address. Two pixels have been stored in parallel in two memories using the same address. Referring to FIGS. 3A, 3B, and 3C, pixel 0 and pixel 1 are stored at the same time at the same address in first memory device 350 and second memory device 375, respectively. Accordingly, for example, pixel 0 is at address 0 in first memory device 350, pixel 1 is at address 0 in second memory device 375, pixel 2 is at address 1 in first memory device 350, pixel 3 is at address 1 in second memory device 375, and so on.
To retrieve pixel data, memories 410 and 415 are put in read mode and address multiplexor 445 is set to connect second input 455 to output 460. Video destination 420 provides an address to destination address bus 450. Destination address bus 450 provides the address to second input 455 of address multiplexor 445. Address multiplexor 445 in turn provides the address to first memory 410 and second memory 415. First memory 410 provides the pixel data stored at the address supplied by address multiplexor 445 from video destination 415 to first data bus 425. Second memory 415 provides the pixel data stored at the same address to second data bus 430. First data bus 425 provides its pixel data to video destination 420 and second data bus 430 provides its pixel data to video destination 420. Two pixels have been retrieved in parallel from two memories using the same address. Referring to FIGS. 3A, 3B, and 3C, pixel 0 and pixel 1 can be retrieved at the same time using the same address from first memory device 350 and second memory device 375, respectively.
FIG. 5 is a block diagram of another implementation of a dual pixel frame buffer architecture 500. Architecture 500 is similar to architecture 400 of FIG. 4, but a memory controller 545 provides data and addresses to memories 510 and 515. Memory controller 545 receives pixel data from video source 505 to store in memories 510 and 515. Memory controller 545 retrieves pixel data from memories 510 and 515 and provides the pixel data to video destination 520. Memory controller 545 replaces address multiplexor 445. Memory controller 545 receives signals from video source 505 and video destination 520 indicating whether pixel data is to be stored to or retrieved from memories 510 and 515. Memory controller 545 generates addresses and supplies these addresses along with control signals to memories 510 and 515. Accordingly, memory controller 545 controls address generation rather than video source 505 and video destination 520, as compared with architecture 400 of FIG. 4. In addition, as noted above with respect to FIG. 4, architecture 500 operates based on clock cycles so that pixel data can be processed for two pixels per clock cycle in support of the desired pixel rate.
6. Double-Buffering
Typical frame buffer architectures often also utilize xe2x80x9cdouble-buffering.xe2x80x9d Double-buffering is a well known technique where the memory address space of a frame buffer is divided into two sections. In some architectures, each section is a separate memory device, and in other architectures one or more devices are each divided into sections. Data from a frame is stored in one section while data from a previously stored frame is read from the other section. Series of reading and writing operations alternate. For example, after storing pixel data for 16 pixels, pixel data for 16 pixels is retrieved. After storing a frame, the sections switch roles. Pixel data for blocks of pixels can be temporarily stored before being sent to memory or after being received from memory in a buffer, such as a FIFO buffer. In architectures 400 and 500 from FIGS. 4 and 5, respectively, FIFO buffers can be included in both the video source and the video destination, or in the memory controller.
The present invention provides methods and apparatus for storing and retrieving data in parallel but in different orders. In one implementation, data for pixels is stored according to a checkerboard pattern, alternately between two memory devices, forming a checkerboard buffer. In one implementation, a checkerboard buffer includes: a data source, providing data in a first order; a data destination, receiving data in a second order; at least two memory devices, each memory device having a plurality of memory locations, where data is stored in parallel to the memory devices and retrieved in parallel from the memory devices, and where data is stored according to the first order using blocks of memory locations, each block having a number of memory locations equal to a power of 2; a first data switch connected to the data source and each of the memory devices, where the first data switch controls which data is stored to which memory device; and a second data switch connected to the data destination and each of the memory devices, where the second data switch controls providing data to the data destination according to the second order.
In another implementation, a checkerboard buffer includes: a video source providing pixel data for pixels in a frame; a video destination; a first memory; a second memory; a first data bus connected to the first memory; a second data bus connected to the second memory; a first address multiplexor connected to the first memory; a second address multiplexor connected to the second memory; a source address bus connected to the video source, the first address multiplexor, and the second address multiplexor; a first destination address bus connected to the video destination and the first address multiplexor; a second destination address bus connected to the video destination and the second address multiplexor; a first data switch connected to the first data bus, the second data bus, and the video source, where the first data switch is between the video source and the first memory and between the video source and the second memory, and where the first data switch switches which memory to a store pixel data for two pixels with each horizontal row of pixels; and a second data switch connected to the first data bus, the second data bus, and the video destination, where the second data switch is between the video destination and the first memory and between the video destination and the second memory, and where the second data switch switches the order pixel data from each of the first memory and the second memory is provided to the video destination with each vertical column of pixels; where pixel data is stored in the first memory and the second memory according to horizontal rows of pixels in the frame, and pixel data is stored in blocks of memory locations, each block having a number of memory locations equal to a power of 2.
In another implementation, a method of storing pixel data in a checkerboard buffer includes: storing pixel data for a first pair of pixels at a first memory address in a first memory device and a second memory device respectively, where the first pair of pixels are the first two pixels in a first horizontal row of pixels in a frame; and storing pixel data for a second pair of pixels at a second memory address in the second memory device and the first memory device respectively, where the second pair of pixels are the first two pixels in a second horizontal row of pixels in a frame and are vertically adjacent to the first pair of pixels; where pixel data is stored in the first memory and the second memory according to horizontal rows of pixels in the frame, and pixel data is stored in blocks of memory locations, each block having a number of memory locations equal to a power of 2.
In another implementation, a method of retrieving pixel data from a checkerboard buffer includes: retrieving pixel data for a first pair of pixels from a first memory address in a first memory device and from a second memory address in a second memory device respectively, where the first pair of pixels are the first two pixels in a first vertical column of pixels in a frame; and retrieving pixel data for a second pair of pixels from the first memory address in the second memory device and from the second memory address in the first memory device respectively, where the second pair of pixels are the first two pixels in a second vertical column of pixels in a frame and are horizontally adjacent to the first pair of pixels; where pixel data is stored in the first memory and the second memory according to horizontal rows of pixels in the frame, and pixel data is stored in blocks of memory locations, each block having a number of memory locations equal to a power of 2.
In another implementation, a method of storing and retrieving pixel data in a checkerboard buffer includes: storing pixel data for a first pixel and a second pixel at a first memory address in a first memory device and a second memory device respectively, where the first pixel and the second pixel are the first two pixels in a first horizontal row of pixels in a frame; storing pixel data for a third pixel and a fourth pixel at a second memory address in the second memory device and the first memory device respectively, where the third pixel and the fourth pixel are the first two pixels in a second horizontal row of pixels in a frame, and the third pixel and the fourth pixel are vertically adjacent to the first pixel and the second pixel, respectively; retrieving pixel data for the first pixel and the third pixel from the first memory address in the first memory device and from the second memory address in the second memory device respectively, where the first pixel and the third pixel are the first two pixels in a first vertical column of pixels in a frame; and retrieving pixel data for the second pixel and the fourth pixel from the first memory address in the second memory device and from the second memory address in the first memory device respectively, where the second pixel and the fourth pixel are the first two pixels in a second vertical column of pixels in a frame and the second pixel and the fourth pixel are horizontally adjacent to the first pixel and the third pixel, respectively; where pixel data is stored in the first memory and the second memory according to horizontal rows of pixels in the frame, and pixel data is stored in blocks of memory locations, each block having a number of memory locations equal to a power of 2.